ci: Add testing using Jenkins pipeline by jlebon · Pull Request #667 · coreos/coreos-assembler

jlebon · 2019-07-25T17:02:58Z

Start running tests in CentOS CI. We build it, run the unit tests, and
in parallel run through a simplified version of the FCOS pipeline
including running kola on the built image.

This is just a start of course. In the future, we could e.g. test each
command more deeply, such as lockfile semantics, overrides, etc...

This uses the projectatomic-ci namespace for now, where we have
resources and permissions set up. Once we're willing to commit more to
this workflow, we can move to a coreos-ci namespace (and let
projectatomic-ci die peacefully).

We're not doing that anymore, so clean things up to make it easier to hack on. Also drop the corresponding el7 Dockerfile.

I'd like the `make check` and `make unittest` targets to be separate stages from building for CI purposes. So just drop that out of the build script, and make it part of the Dockerfile itself so we're still running the tests when building the image.

For building, we were putting `cmdlib.sh` in the root of the container build dir. But I want to be able to use `build.sh` from a git checkout, so rebuild that same layout. Also drop the direct reference to `/root/containerbuild` for the same reason. And also add the motivation for why the `Dockerfile` calls `build.sh` in multiple steps like this.

Allow calling `build.sh` without any arguments to be equivalent to going through all the build steps one after the other. This will be used in a CI context.

Start running tests in CentOS CI. We build it, run the unit tests, and in parallel run through a simplified version of the FCOS pipeline including running `kola` on the built image. This is just a start of course. In the future, we could e.g. test each command more deeply, such as lockfile semantics, overrides, etc... This uses the `projectatomic-ci` namespace for now, where we have resources and permissions set up. Once we're willing to commit more to this workflow, we can move to a `coreos-ci` namespace (and let `projectatomic-ci` die peacefully).

ajeddeloh

Mostly minor nits. I am no jenkins guru so I don't have much to say about the last commit. Otherwise LGTM

ajeddeloh · 2019-07-25T21:14:27Z

build.sh

-else
-    echo 1>&2 "should be on either RHEL or Fedora"
+if ! grep -q '^Fedora' /etc/redhat-release; then
+    echo 1>&2 "should be on either Fedora"


s/either Fedora/Fedora

ajeddeloh · 2019-07-25T21:15:36Z

build.sh

+    # Add continuous tag for latest build tools and mark as required so we
+    # can depend on those latest tools being available in all container
+    # builds.
+    echo -e "[f$release-coreos-continuous]\nenabled=1\nmetadata_expire=1m\nbaseurl=https://kojipkgs.fedoraproject.org/repos-dist/f$release-coreos-continuous/latest/\$basearch/\ngpgcheck=0\nskip_if_unavailable=False\n" > /etc/yum.repos.d/coreos.repo


Could replace this with a heredoc to make the \n's a little easier to understand, but that can wait for a different PR if you want.

ajeddeloh · 2019-07-25T21:16:19Z

build.sh

-    if [ -n "${ISFEDORA}" ]; then
-        rpm -q grubby && yum remove -y grubby
-    fi
+    rpm -q grubby && yum remove -y grubby


stale comment

cgwalters

Looks great to me!

cgwalters · 2019-07-25T22:09:32Z

ci/pod.yaml

+kind: Pod
+spec:
+  # XXX: tmp hack to get anyuid SCC; need to ask to get jenkins SA added
+  serviceAccountName: papr


Fine for now, though if we start spreading this across multiple projects I think this is the kind of thing we'll want to abstract out.

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - checks that it can build an RPM - builds FCOS (with the new binaries both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. Let's address those trickier parts as follow-ups. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131

This is an experiment in using Jenkins pipelines for our CI. See similar initiatives in coreos-assembler[1] and fedora-coreos-config[2]. For now, this only does the following testing: - checks commit for unintended submodule bumps - checks the minimum Rust version - builds RPMs - builds FCOS (with the new RPMs both for executing the build itself, as well as included in the built OS) There are dummy placeholders for where we'd actually run the vmcheck and the compose testsuites. Let's address those trickier parts as follow-ups. [1] coreos/coreos-assembler#667 [2] coreos/fedora-coreos-config#131 Closes: #1899 Approved by: cgwalters

jlebon force-pushed the pr/jenkins branch from 03b33f6 to 8c1bf6a Compare July 25, 2019 17:54

jlebon added 5 commits July 25, 2019 14:06

Drop more el7 conditionals

77681a2

We're not doing that anymore, so clean things up to make it easier to hack on. Also drop the corresponding el7 Dockerfile.

build.sh: Allow running without arguments

0acc59a

Allow calling `build.sh` without any arguments to be equivalent to going through all the build steps one after the other. This will be used in a CI context.

jlebon force-pushed the pr/jenkins branch from 8c1bf6a to 56d05a0 Compare July 25, 2019 18:07

ajeddeloh approved these changes Jul 25, 2019

View reviewed changes

cgwalters reviewed Jul 25, 2019

View reviewed changes

cgwalters merged commit eec7ddf into coreos:master Jul 26, 2019

cgwalters mentioned this pull request Aug 29, 2019

CI: Prow + "CoreOS CI" tracker coreos/fedora-coreos-tracker#263

Closed

jlebon mentioned this pull request Sep 6, 2019

ci: Add Jenkins pipeline coreos/rpm-ostree#1899

Closed

jlebon deleted the pr/jenkins branch July 6, 2020 20:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: Add testing using Jenkins pipeline#667

ci: Add testing using Jenkins pipeline#667
cgwalters merged 5 commits intocoreos:masterfrom
jlebon:pr/jenkins

jlebon commented Jul 25, 2019

Uh oh!

ajeddeloh left a comment

Uh oh!

ajeddeloh Jul 25, 2019

Uh oh!

ajeddeloh Jul 25, 2019

Uh oh!

ajeddeloh Jul 25, 2019

Uh oh!

cgwalters left a comment

Uh oh!

cgwalters Jul 25, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jlebon commented Jul 25, 2019

Uh oh!

ajeddeloh left a comment

Choose a reason for hiding this comment

Uh oh!

ajeddeloh Jul 25, 2019

Choose a reason for hiding this comment

Uh oh!

ajeddeloh Jul 25, 2019

Choose a reason for hiding this comment

Uh oh!

ajeddeloh Jul 25, 2019

Choose a reason for hiding this comment

Uh oh!

cgwalters left a comment

Choose a reason for hiding this comment

Uh oh!

cgwalters Jul 25, 2019

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants